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Abstract 

We argue that the time derivative in a fixed coordinate frame may not 
be the most appropriate measure of time regularity of an optical flow held. 
Instead, for a given velocity held v we consider the convective acceleration 
vt-\-Vvv which describes the acceleration of objects moving according to 
V. Consequently we investigate the suitability of the nonconvex functional 
\\vt -\- Vvv\\\2 as a regularization term for optical how. We demonstrate 
that this term acts as both a spatial and a temporal regularizer and has 
an intrinsic edge-preserving property. We incorporate it into a contrast 
invariant and time-regularized variant of the Horn-Schunck functional, 
prove existence of minimizers and verify experimentally that it addresses 
some of the problems of basic quadratic models. For the minimization we 
use an iterative scheme that approximates the original nonlinear problem 
with a sequence of linear ones. We believe that the convective acceleration 
may be gainfully introduced in a variety of optical how models. 


1 Introduction 

Motivation. Optical how is the apparent motion in a sequence of images 
and can be described by a velocity held. Using variational techniques to esti¬ 
mate this velocity held requires the design of an appropriate energy functional. 
Typically such a functional is a sum of two parts. The hrst part, sometimes 
called data term, measures the accuracy with which the velocity held describes 
the observable image motion. By ensuring consistency of the how held, the 
second part, also called regularization term, gives information the data term 
cannot provide and thereby addresses the inherent ill-posedness of the optical 
how problem. Naturally, related research has to a large degree been concerned 
with tuning the second term to optimally capture the characteristics of velocities 
of real-world image sequences. 

Generally speaking regularization terms for optical how fall into two cate¬ 
gories: those which only penalize spatial derivatives of the velocity held, and 
those where derivatives in both space and time are penalized. The hrst category 
is by far the more popular one. Its developments have closely followed those 
of variational image restoration, because the regularity of images dictates to a 
signihcant degree the spatial regularity of velocity helds describing their motion. 
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In other words, image discontinuities tend to coincide with motion discontinu¬ 
ities. These, it was found, can be appropriately dealt with using subquadratic 
or anisotropic regularization. 

Research on the second category, i.e. that of spatiotemporal regularization, 
is relatively scarce. Again this is in analogy with the field of image process¬ 
ing, where the joint restoration of image sequences is quite under-represented 
as compared to the restoration of single frames. The inclusion of time deriva¬ 
tives has the obvious disadvantage of turning a series of decoupled 2D problems 
into one 3D problem. In the early days of computer vision limited computer 
memory and processing power was prohibitive to such approaches. Clearly this 
is no longer the case. On the contrary, recent publications have articulated and 
addressed the need for time-coupled models even for data with three instead of 
two space dimensions. See for example HIE]. 



Figure 1: Using partial time derivatives in flow regularization can blur across 
object boundaries. Here, the shaded area represents the space-time signature 
of a moving object and the time direction points both inside and outside of it 
at different points. 

It is well-known that indiscriminate smoothing in space can lead to blurring 
at the boundaries of objects. This statement is equally true in the space-time 
domain, if we identify objects with their space-time signature (Figure [^. In 
particular, blind penalization of the partial time derivative of a velocity field 
can lead to a loss of accuracy, especially at the boundaries of moving objects. 
Therefore we suggest to take the time derivative only along the movement of 
the object. 

This derivative has a very natural physical interpretation. Let v{t,x) be a 
time dependent velocity field on and let t be the trajectory of 

a certain object initially located at xq G that moves according to v. The 
translation into mathematical terms of this connection between 7 and v is the 
following initial value problem 

'y'{t;xo) =v{t,^{t;xo)) 

7 ( 0 ;a:;o) = xq. 

Taking time derivatives on both sides of the differential equation above gives a 
formula in terms of v for the acceleration of the object moving along 7 

'l''it;xo) = ^v{t,'y{t;xo)) 

= ^0)) + 7(t; xo)) v{t, 7(t; xq)) 
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This expression, which is sometimes called convective acceleration^ is to be con¬ 
trasted with the partial time derivative of v evaluated at (t,7(t;xo)) 

— v{t,j{t]Xo)), 

which has a notably different physical meaning. In this article we argue that 
in some situations the former can be more appropriate for the regularization of 
optical flow. 

This work is devoted to investigating the suitability of the functional 

V 1-^ \\vt + Vvv\\l 2 (2) 

as a smoothness term for optical flow. We believe a sensible way to do so is to 
include this term into a simple optical flow model and to compare the results 
with those of the same model without the new term. The reference model we 
choose for this task is a contrast invariant and time-regularized version of the 
Horn-Schunck functional [7]. Incorporating (© into our model comes, however, 
at the cost of leading to a nonconvex functional and to nonlinear optimality 
conditions. Numerically, we approximate the problem by a sequence of convex 
quadratic ones. 

We stress that it is not the aim of the proposed model to compete in the 
highest ranks of motion estimation benchmarks. Instead we want to point out 
a certain aspect of optical flow regularization, which we feel has not received 
its due amount of attention. Even though it leads to a more challenging varia¬ 
tional problem, we demonstrate that a reasonable approximation is numerically 
tractable and pays off in terms of accuracy. 

Related work. Horn and Schunck [7] are generally credited with having laid 
the groundwork for variational optical flow models. Penalizing the squared 
L^-norms of the spatial derivatives, their model regularizes the optical flow 
isotropically as well as homogeneously over the image domain. In uni Nagel 
proposed to suppress smoothing across image discontinuities by only penalizing 
the derivatives of the vector field along the level lines of the image. This idea 
was subsequently extended to the space-time domain in m- 

A different route to spatiotemporal regularization was taken by Weickert and 
Schnorr. In m they used an isotropic but essentially subquadratic regularizer. 
While still convex this approach leads to nonlinear optimality conditions. In 
the survey [18] the same authors classify convex spatial as well as spatiotempo¬ 
ral regularizers for optical flow. According to their nomenclature our approach 
would classify as anisotropic and flow-driven. It is, however, fundamentally dif¬ 
ferent from the regularizers considered there, not only because of nonconvexity. 
In [IHI the anisotropy of flow-driven regularizers is determined by the Jacobian 
of the velocity field, whereas in our approach it is determined by the velocity 
itself. 

Chaudhury and Mehrotra pursue an interesting trajectory-based regulariza¬ 
tion strategy [3] . Motivated by the principle of least action and the inertia of 
motion they postulate that both length and curvature of trajectories of moving 
objects should be minimal. There is a close connection between a trajectory’s 
curvature and the convective acceleration of an object moving along that tra¬ 
jectory. We discuss this connection in the next section. More recently, Salgado 
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and Sanchez M as well as Volz and coauthors m have proposed time-discrete 
models using explicit trajectorial regularization. The present article tries to 
capture the essence of trajectorial regularization for optical flow in an entirely 
continuous setting. 

Finally, we note that for the problem of image sequence reconstruction it 
is not uncommon to regularize the sequence along the optical flow, which can 
be either precomputed or estimated simultaneously. This derivative along the 
optical flow is just the convective derivative of the image sequence. See mn] 
for example. 

Outline. In Section [2] we introduce our model. In Sec. 12.II we discuss the 
convective acceleration of vector fields. Section |2.2| contains properties of the 
functional \\vt + 'Vvv \\‘^2 as a regularization term. The last part of Sec. [^treats 
our choice of data term. Section is dedicated to discussing our numerical 
minimization approach. Finally, we present experimental results in Section 


2 Model 


2.1 Convective acceleration 


Notation. Let c be a bounded Lipschitz domain, T > 0, and set 
E = (0, T) X li. We denote points in by a; = . Let cf) : E ^ Q.he 

a flow on Vt\ That is, for fixed t G (0,T) the map •) is a diffeomorphism of 
while for fixed xq G ^2 the trajectory described by 0(-,xo) is smooth. The 
vector field on E that gathers the velocities of all trajectories associated to (j) is 
defined by 


u{t,x) = (f>t{t,Xo) 


xo=(f> 


(3) 


where (pt is the partial derivative of p with respect to its first argument and 
denotes the inverse of 0(t, •) evaluated at x. Notice that u{t,x) de¬ 
scribes the velocity of the trajectory passing through x at the time t, so u{s^ x) 
in general corresponds to a different trajectory, \i s 

Let f : E ^ be a possibly vector-valued quantity on E. We define the 
convective derivative of / along p as 


Duf{t, x) = a;o)) 

CLi xo=(p~^{t,a 

= ft{t, x) + V/(t, x)u{t, x). 


(4) 


Here V/ = G is the spatial gradient of /. Notice that the 

convective derivative only depends on u(t^x)^ thereby justifying the notation 
Duf‘ Using the notation V/ = {ft, V/) and u = we can write D^f = 

V fu. If we set f = u, the resulting vector field 

DuU = Ut E Vuu = Vuu (5) 


is called the convective acceleration of the flow p. 


4 




Flows with vanishing convective acceleration. When proposing a new 
regularization term, it is always helpful to know when it is minimal. Therefore, 
we briefly discuss what conditions must be met so that a vector field u satisfies 


DuU = 0 , for all (t, x) G 


(6) 


and give a few examples afterwards. Intuitively DuU = 0 means that u satisfies 
a “flow constancy condition”. This is to be understood as an analogy to the 
widely known brightness constancy condition^ which is frequently used to derive 
the optical flow equation. Note that with the notation introduced in the previous 
paragraph, for an image sequence /, the optical flow equation reads Duf = 0 . 

From the definition of the convective derivative 0 and 0 it follows that 

DuU{t,(f>{t,Xo)) = (ptt{t,Xo), 

for all xo G and t G (0,T). Now fix an xq G O and, for simplicity, denote the 
trajectory (/)(•,xq) by 7 (-), so that we have D^u = 7 ". Assuming that 7 ' does 
not vanish on (0,T), we can write 



Since the unit tangent vector 7 VI 7 I is always perpendicular to its derivative, 
the convective acceleration 7 " is a sum of two mutually orthogonal vectors. 
Such a sum can only vanish, if both vectors vanish. Due to the assumption that 
7 ' 7 ^ 0 , we conclude that 



This proves the quite intuitive fact that, if a flow has vanishing convective 
acceleration, then all its trajectories must be straight lines and have constant 
speed. Below we give a few examples of such vector fields. 

Example 1. Let : M ^ M be a differentiable function. The two vector fields 
vi^V 2 E defined by 



do not depend on time and satisfy VviVi = 0. Thus, by formula 0 they have 
vanishing convective acceleration. Their integral curves are parallel to the X 2 - 
and xi-axes, respectively. 

Example 2 . Let 0^0. Denote by the unit vector in radial direction and 
let ^ : M ^ M be a differentiable function with period 27r. The vector field 
'z; : ^ which in polar coordinates is given by 


= g{>f)er 


has zero convective acceleration. In contrast to the previous example the integral 
lines of v are not all mutually parallel. 
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Example 3 . The vector fields in the previous examples were all constant in time. 
Time-dependent examples can be constructed from time-dependent solutions u 
of the inviscid Burgers’ equation in one space dimension ([6], Section 3 . 4 ). That 
is, let u : E solve ^ with E = ( 0 , T) x M. Then 


vi{t, 




V2{t,X^,X^) = ^ 


0 

u{t^ x‘^) 


satisfy Dy.Vi = 0 while having partial time derivatives that do not vanish iden¬ 
tically. As in the first example the integral curves are mutually parallel straight 
lines. The main difference, however, is that two trajectories passing through the 
same point in space but at different times, might do so at different speeds. 


Relation to curvature. Given a curve 7 : ( 0 , T) ^ we can define its 
curvature as the normal part of the arclength derivative of the unit tangent 
vector 

_ / Y \ ^ 1 d Y _ Y^ -7" 

^"“VlYlJ 'lYI^^YYr lYP ’ 

where (a, 6)"''-*- = (—6, a)"'". If 7 is arc length parametrized, that is Y'j = 1 , then 
the expression simplifies to = |7"|, because in that case 7'-*- = 7^Vl7"l- 

For a given xq G Q we can apply the formula above to the trajectory t i-^ 
0 (t, xq) of a flow (j). Recall that in this case 7' becomes u and 7" becomes DuU. 
Thus we get 

u-^ • DuU 

f^cf)(-,xo) = ^ • 

In particular, whenever \u{t, xo))\ = 1 on some subinterval of ( 0 ,T), then 

the norm of the convective acceleration \Duu\ coincides with the absolute value 
of the curvature of on this subinterval 

2.2 Convective regularization 

In this section we study some properties of the functional 

MlH. (7) 

as a regularization term for optical flow. By || • \\l2 we mean the norm of 
Below we continue to use this shorthand whenever convenient. 

luterpretatiou of the couvective terui. Functional § has two interpre¬ 
tations, one as a smoothing term and and another one as a projection term. 
Recalling the bar-notation we introduced in Section |2.1[ the integrand of 0 
can be written as a quadratic form in the partial derivatives of u 

|D„n|2 = (8) 

Observe that the diffusion tensor 
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is a projection matrix onto the span of u composed with a scaling by 
Therefore, minimization of 0 leads to smoothing of the vector field u only in 
direction fx, where precedence is given to regions where the magnitude of u is 
relatively large. Clearly, since u ^ ( 1 , 0 , 0 )^ in general, the proposed convective 
regularization term is not a purely temporal regularizer, but it also enforces 
spatial smoothness of the vector field in a way that is consistent with the motion. 
Figure [^illustrates this behavior. 



Figure 2: The shaded area represents the space-time signature of a moving 
object. The arrows not only indicate the space-time velocity u of the object but 
also the direction of diffusion enforced by the tensor uu^. As opposed to the 
situation with the time derivative (Figure Q, object boundaries are respected. 

On the other hand, the roles of u and Vxx* can be exchanged so that (j^ 
rewrites as a sum of two quadratic forms in u 

\Duuf = Wfx + Wfx. ( 9 ) 

The matrix (Vxx*)^ Vxx* projects onto the span of Vxx* and scales by the squared 
magnitude of Vxxb From this point of view minimizing 0 amounts to forcing 
u to be orthogonal to the directions of greatest change of both and where 
precedence is again given whenever this change is relatively drastic. Note that 
the outer product of a vector with itself is always a positive semidefinite matrix, 
so that there can be no cancellations in (|^, (j^. 

The concurrence of the two different interpretations above is made explicit 
by looking at the Euler-Lagrange equation of 0 . given by 

u-V- {Wuuu'^) = 0. (10) 

Here, the space-time divergence operator V • acts along rows. 

Variational properties. In the following we adopt the notation A < B when¬ 
ever A < B holds up to multiplication by a positive constant. According to the 
basic estimate 

\\Duu\\h<\\u,\\h + \\u\\U\Vu\\h 

a seemingly natural space over which to minimize, for a given image sequence 
/, the optical flow functional 

T{u) = \\Duf\\l.+a\\Duu\\h 

would be X = M^) nI/^(E, M^). Now, in order to show existence of min- 

imizers using the direct method [ 5 ] , we need to ensure existence of a minimizing 
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sequence which converges. This is typically done by establishing a coercivity 
condition of the type 

hrx-b 

for some 6 > 0 and p > 1 . In the examples above we have seen, however, 
that there are vector fields u with vanishing convective acceleration but nonzero 
partial derivatives. This implies that not even the inequality 

\\Duu\\h> ||V<.-6 

holds true. Therefore, in order to guarantee existence of minimizers we consider 
the functional 


£ ; [0,+oo] 

£{u) = WXDufWh + a\\Duu\\l, + l 3 \\Vu\\l„ 


( 11 ) 


where a ,/3 > 0 and A 
Sec 


2.3 


E IR+ is a weighting function which is specified in 
In addition we have to make the assumption introduced in [ 16 ] that 


the partial derivatives of / are linearly independent in L‘^{E)^ that is 


(/®i)/®2 )l2 < ||/a:i||LHI/a:HlL2- (12) 

Proposition 1. Let f G satisfy ( [T^ and assume that A G L^{E) is 

sueh that ess inf A > 0 . Then the minimization of £ over has at least 

one solution. 


Proof We show that £ is proper, coercive and lower semicontinuous with respect 
to the weak topology of H^{E, M^). That £ is proper follows from nonnegativity 
and the assumptions on / and A. 

Next we prove the coercivity estimate 

£iu)>\\ufH.-b 

for b > 0 . We obviously have £(u) > /?||Vu|||2- So it remains to show that 
Z ll'^^llia “ Denoting the average of u hy ue = 1 /|£^| Je ^ have 

Nli^<IKIIi2 + ||u-«^lli2 

<\\nE-Vf\\l. + \\Vu\\l. 

< Ik ■ V/lli^ + ||(« - ue) ■ VfWh + Mh (13) 

<l|D„/iii. + ii/tiii. + i|vu|ii. 

<^(«) + ll/tlli=, 

which proves coercivity. There are three main ingredients in the above estimate. 
The first one is a quadratic variant of the triangle inequality 

||u + u;||2<2||u||2 + 2||u;||2. 


The second one 

\\uE\\h<\\uE-yf\\h 


uses the assumption of linear independence of f^i and fx2 and is proved in [T6l 
p. 29 ]. The third ingredient is the Poincare inequality \\u — ue\\l^ ^ ||Vrt||L2 
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(El, Theorem 8.11). Also note that the fourth inequality in ( [T^ requires the 
assumption / G while the last one uses ess inf A > 0 . 

Let {un} C converge to u in the weak topology of In 

particular, Vun converges weakly in to the corresponding gradient 

Vu. By the compact embedding of into up to choosing a 

subsequence we may assume that Un also converges strongly in to u. As is 
made apparent in (|^, the expression \Duu\^ is convex in Vu for fixed and £ 
is always nonnegative. We can therefore apply a standard result ([5], Theorem 
3.23) to obtain that £ is lower semicontinuous. □ 


There is no reason to expect minimizers of £ to be unique. The following 
example demonstrates that £ is not convex in general. 

Example 4. Let vi^V 2 be the two vector fields from Example with g being the 
identity, that is 


vi(t^ x^) 



V2{t,X^,x‘^) 


They satisfy Dy^vi = Dy^V 2 = 0, while 




1 
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where w = {vi V 2 )/2. We conclude 

0 = ||-D,,iVi||i 2 = \\Dy^V 2 \\l 2 < \\D^w\\l 2 = ^ jjx\^dx 


SO that the functional u cannot be convex. This clearly implies 

that there are (3^ f such that £ is not convex. 


2.3 Data term and contrast invariance. 


Contrast invariance is a useful property of image processing operators. Let h 
be a change of contrast, that is, a differentiable real function with h' > 0. An 
operator A is called contrast invariant, if it commutes with all such contrast 
changes: 

Aoh{f) = hoA{f) 
for all images / ([I], Sec. 2.3). 

A similar property can be postulated for operators estimating image motion, 
since an order-preserving rearrangement of the grey values of an image sequence 
should certainly not change velocities. Therefore, we call an operator A that 
maps image sequences / to velocity fields contrast invariant^ if it satisfies 

A{f) =Aoh{f). 


However, a typical optical flow model of the form 

/ i-> argmin{||L>„/||^p + aTZ{u)} 


(14) 


does not have this property. A simple counterexample is multiplication by a 


positive number h{f) = cf. For a model like (14) this change of contrast effec¬ 
tively amounts to dividing the regularization parameter a by and therefore 
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definitely influences the result. There is, however, also a local effect of this 
contrast dependence, which is easy to confirm experimentally. Consider a scene 
with a dark background and two moving objects which are similar in shape and 
size and which move at similar velocities, but which have significantly differ¬ 
ent average brightnesses. This is precisely the case in the example of Figure 
Then, the velocities of the darker object, i.e. the one with lower values of /, will 
be regularized more strongly than those of the brighter one, if a data term like 
the above is used. 


C| c 

^ ol ® • 


Figure 3: Effect of the data term weighting for £ with a = 0. When it is omitted, 
the data term inappropriately favors high-contrast objects. Shown are, from 
left to right, one frame of /, the corresponding results without weighting and 
P = 0.2, without weighting and (3 = 0.5, and weighted with uj = \/|V/p + 
and P = 0.005. 

However, contrast dependence is not an inherent property of optical flow. 
Observe that, if u solves Duf = 0, then it also solves Du{h of) =0, because 
Du{h o /) = h'Duf and h' > 0 by assumption. Therefore, it should rather 
be regarded a side-effect of the variational regularization approach. Contrast 
invariance can be restored by weighting the data term with A = l/cj, where 
u; is a positively 1-homogeneous function that depends only on the first order 
derivatives of /. It then follows that 

Dujhof) ^ h'Duf ^ Duf . . 

uj{V{hof)) h'uiVf) uiVfY ^ ^ 

Such weighted data terms have already been used, although it seems with 
a different reasoning. In [3 [JO], for example, the weight was chosen to be cj = 
y^lV/p + e^. In this paper we experiment with the weight uj = a/|V/ p + e^. 
With this choice the data term essentially measures the orthogonal projection 
of u onto the span of V/, since 



^ 5' ft' 


Duf 

UJ 


V/ _ 

u. 


|v/| 


The effectiveness of this choice is illustrated in Figure in which the relative 
velocities of the vehicles are recovered correctly even though they have widely 
different contrast levels. 

Finally, observe that weights which do not depend on first derivatives of / 
(only), like u; = |/| + e for instance, do not yield full contrast invariance. While 
they work fine for the special case of rescaling h{f) = c/, they fail in general 
since the factor h' in (15) does not cancel. 
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3 Numerical solution 


We now seek to formulate a numerical approach for the minimization of the 
functional £. As shown above, the convective regularization term is nonconvex 
and leads to nonlinear optimality conditions. Therefore, we propose an iterative 
scheme to arrive at an adequate flow field. 


Iterative scheme. Consider the functional 


g{u,w) = ||AD„/||i. +a||D^«||i. +^||Vw||i., 

which satisfies £{u) = Q{u^u). For fixed w G the mapping u i-G 

Q{u^w) is a convex and quadratic functional, and Q{u^w) < +oc for any u G 
Our iterative method then reads as follows. 

• Given an image sequence / and parameters Po^ai^j3i\ 

• Find an initial guess uq by minimizing Q{u^ 0) over with a = 0, = /do. 

This induces isotropic regularization in time and space, corresponding to 
a time-regularized variant of the Horn-Schunck model. 

• Compute Uk^i by minimizing over the variable u the functional Q{u^Uk) 
with parameters a = p = Pi. 


Each of these steps corresponds to finding an optical flow field using anisotropic 
regularization with the diffusion tensor (atDtD^T/dld, where Id is the 3x3 identity 
matrix. Therefore, this particular scheme is based on the diffusion interpretation 
of the convective regularization term, as reflected in (§. Indeed, upon fixing w 
the first term in the complete Euler-Lagrange equation (10) does not appear. 


Even though convergence of the iterative scheme is not guaranteed, in all of our 
experiments the solution Uk stabilized after only a few iterations. Typically by 
/c = 3 or 4 the difference between successive iterates became negligible. 

By the properties of Q listed above, each of these minimization problems can 
be performed by solving the corresponding linear Euler-Lagrange equation 


(yu{aww^ 


+ /3Id)) - A2(V/ • u)Vf = X^ftVf 


(16) 


coupled with natural boundary conditions 


, • (yu{aww~^ + /^Id)) =0 on 9E, 


where n is the outward normal to E. Our approach of discretizing (16) is 
specified next. 


Discretization. The model has been discretized using a multilinear finite 
element formulation on a regular rectangular grid, so that each vertex in the 
grid corresponds to one pixel of one video frame. The grid used is such that the 
spacing between nodes in space is one, while the spacing in time (from frame 
to frame) may be smaller. We emphasize that the relation of the two grid step 
sizes implicitly controls the relative amount of regularization done in time and 
space, since the derivatives of the functional scale accordingly. Usually, using 
the same spacing will lead to results that are highly over-regularized in time. In 
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the presented examples, the time step was divided by a factor of 8 with respect 
to that of space. 

Both the input / and the unknown vector field u are represented by their 
nodal values, leading to continuous piecewise multilinear functions. Since the 


Euler-Lagrange equation (16) is actually a system for the two components ^v? 


the resulting linear system is represented by a block matrix, with the different 
blocks describing the interactions between components. The entries of these 
blocks are assembled from the integrals over different elements arising in the 
weak formulation, as per finite element practice m- Such integrals are com¬ 
puted exactly through an adequate Gauss quadrature. Note that boundary 
conditions do not need to be imposed explicitly. 

A consequence of representing / by a piecewise multilinear function is that 
the corresponding partial derivatives f^i , ft appearing in the right hand side 
A^/tV/^ of (16) do not belong to the same finite element space as /. However, 


our finite element matrices act on the nodal values of functions in this space, 
so an interpolation step is needed to obtain functions with the appropriate 
smoothness. To this end, we have performed an projection of these partial 
derivatives into the space of piecewise multilinear functions, whose nodal values 
are then multiplied together and used as the right hand side for our linear 
system. Computationally each projection requires the solution of an additional 
linear system, which needs to be done only once. 


4 Experiments 

In Figures 1^1^ and 1^ we present some examples of flows computed through the 
numerical scheme of the previous section, approximating minimizers of 

£{u) = ||AD„/||i. +a||D„u||i. +^||Vu||i., 

which are compared with corresponding results with basic isotropic regulariza¬ 
tion, that is, minimizers of 

n{u) = \\\Duf\\h + mM\h- 

In all these examples results were computed over an interval of 30 frames. 

In the sequences used, the time derivative of the vector field at a given po¬ 
sition is likely not appropriate, since isolated objects travel across the images. 
In the results with isotropic regularization, some of the disadvantages of basic 
quadratic regularizers are apparent. With lower regularization parameters (Fig¬ 
ures [4(^ and [5(^ inner edges of rigidly moving objects are visible. With higher 
regularization parameter values, the support of the vector field is enlarged and 
interactions between distinct objects moving ensue, creating cancellations when 
the objects move in different directions (Figure 4(c)| ) or artificial reinforcement 
when they move in similar directions (Figure |5(c)[ ). However, with convective 
regularization both of these disadvantages can be avoided (Figures |4(d)| and 

m- 

Additionally, one needs to choose the regularization constant e for the weight 
v/|V/P + e2 in the data term. Clearly, choosing a too large e will diminish the 
effect of the weighting scheme, while choosing it too small may overemphasize 
very low contrast regions and potentially amplify noise. Since the 8-bit input 
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images were normalized so that their values are of the form /c/255 with k = 
0.. .255, we chose e = 0.01. That is, e is roughly of the size of the smallest 
nonzero derivative that may appear in the data. 



(c) Result with l-L^ (3 = be — 3 



(b) Result with /3 = 5e — 4 


c 


(d) Result with 8^ a = 5e — 3, /3 = 5e — 4 


Figure 4: Comparison of isotropic and convective regularization. 

In our experiments the parameter may be chosen quite small to preserve 
the boundaries of objects, while using a relatively large ai for the convective 
regularization term. By using the convective derivative, the information from 
other frames is used in a consistent way to correctly fill the flow field inside 
objects even in the absence of texture. The initial isotropic parameter /3o for 
starting the iterative scheme was chosen to be (3^ = ai in all cases. 

5 Conclusion 

In this article we presented an entirely continuous optical flow model that is 
based on the assumption that velocities should vary smoothly not at fixed points 
in space but along motion trajectories. The resulting regularization term has a 
variety of different interpretations. First, this term penalizes the proper accel¬ 
eration of objects moving according to the velocity field. Second, it has a direct 
relation to the curvature of motion trajectories. Finally, when minimized it acts 
on the vector field not only as a projection term but also as an edge-preserving 
anisotropic regularizer. 
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Figure 5: Comparison of isotropic and convective regularization. 




Figure 6: Cutout from Figure The use of convective regularization allows to 
obtain nearly-constant optical flow field inside the objects without excessively 
enlarging the support. 
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We tested the convective regularization term by incorporating it into a ref¬ 
erence optical flow model and comparing the results with and without the new 
term. In order to make the changes as apparent as possible and to avoid po¬ 
tential side effects we tried to keep the complexity of the reference model at a 
minimum. Therefore, the resulting functional (11) should mainly be viewed as 
a prototype of an optical flow model with convective regularization which can 
be improved upon by future research. 
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